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Video processing has gained a lot of significance because of its applications 
in various areas of research. This includes monitoring movements in public 
places for surveillance. Video sequences from various standard datasets such 
as I2R, CAVIAR and UCSD are often referred for video processing 
applications and research. Identification of actors as well as the movements 


in video sequences should be accomplished with the static and dynamic 


background. The significance of research in video processing lies in 
Keyword: identifying the foreground movement of actors and objects in video 
; sequences. Foreground identification can be done with a static or dynamic 

Adaptive threshold back d. Thi f identificati ; . 
. ground. This type of identification becomes complex while detecting the 
Background subtraction movements in video sequences with a dynamic background. For 
Frame difference identification of foreground movement in video sequences with dynamic 
Motion detection background, two algorithms are proposed in this article. The algorithms are 
Tracking termed as Frame Difference between Neighboring Frames using Hue, 
Saturation and Value (FDNF-HSV) and Frame Difference between 
Neighboring Frames using Greyscale (FDNF-G). With regard to F-measure, 
recall and precision, the proposed algorithms are evaluated with state-of-art 
techniques. Results of evaluation show that, the proposed algorithms have 

shown enhanced performance. 
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1. INTRODUCTION 

In video processing, foreground identification can be accomplished with a static or dynamic 
background. This process is difficult, when identifying the movements in video sequences with a dynamic 
background. Numerous algorithms have performed the foreground identification in video sequences. It is 
found that, these algorithms have done the foreground identification with less importance to changes in 
background and illumination. This work proposes two algorithms FDNF-HSV and FDNF-G. The proposed 
algorithms have identified the foreground movements with significance to variations in illumination and 
background. The proposed algorithms have addressed the issue, by computing the adaptive threshold of the 
changes among frames. This section presents the literature of the relevant work studied. 

A robust technique for target tracking was presented in [1]. This technique mines the targets from a 
video. Then, the targets were categorized pertinent to properties of images. This method has tracked objects 
with variations in appearance. A background model for surveillance with the Pantilt-Zoom (PTZ) camera was 
proposed in [2]. This model has presented a texture descriptor for encoding spatio-temporal data. This model 
was linked with a set of techniques for tracking objects. In [3], a textual descriptor referred as Center 
symmetric scale invariant local ternary patterns (CS-SILTP) was contributed for building a background 
model. This model was applied for identifying foreground objects in each of the captured frames. To identify 
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the objects in motion, a non-panoramic algorithm relying on spatio-temporal background was presented 
in [4]. This algorithm has solved the problems with panoramic methods such as slow initialization, 
background adaptation and error accumulation. Relying on Bayes decision framework, a technique to excerpt 
foreground objects from a video was contributed in [5]. In this work, a background with moving and static 
objects was modeled. Then, the objects in the foreground were identified by combining the outcome of static 
points and points in movement. 

By applying a mixture model, the work proposed in [6] identifies shadows in motion. This technique 
has reduced overhead by detecting shadows only on foreground pixels. By sampling values, the codebook 
background subtraction method presented in [7] captures structural background movements for a long time. 
This algorithm has tracked moving objects with varying background and changes in illumination. To deal 
with complicated dynamic background, a method was proposed in [8]. This method has proposed a scale- 
invariant pattern operator and a technique for estimation of kernel density in patterns. By combining these, 
the method has handled dynamic background. An algorithm for detecting objects in motion was offered in 
[9]. This method stores the values of pixels in the past. Then, it relates the present and past values of the 
pixel. Finally, the method identifies the particular background to which the pixel belongs. By combining the 
color information and SILTP, a block-wise background model was contributed in [10]. This model has dealt 
with multimodal and dynamic background. For efficient identification of objects in motion, the color 
information and SILTP were combined. 

An efficient method based on background subtraction was presented in [11]. By reducing the data 
dimensionality of image frame and applying the sparse representation, the method has extracted foreground 
objects. A new integrated method was proposed in [12]. By combining the region growing with thresholding, 
the method finds the region of interest (ROD in an image. An algorithm termed as DTIGLMM-K was offered 
in [13]. The algorithm has focused on image segmentation in an enhanced manner. This algorithm was 
appropriate for various types of data and applications. The method developed in [14] presents a framework 
for detection and recognition of human actions. The method segments the objects in motion, excerpts a set of 
features and chooses the features. For recognition of actions, the chosen features were classified with a 
multiclass SVM. A method for identifying moving objects was proposed in [15]. The method assesses the bi- 
directional optical flow between frames. Then it is improved and normalized. Finally, the method identifies 
the moving objects by verifying the threshold of the optical flow of each block and the optical flow of the 
object under consideration. An algorithm termed as ‘Good Features to Track’ was presented in [16]. First, the 
algorithm extracts feature from the frames. Then, the features of moving objects were excerpted from the 
subsequent frame. Based on the motion information and location, the objects in motion for each frame were 
identified. This work has also formed clusters of objects in motion. 

Depending on the fixed spatial association among the current and randomly chosen pixels, a novel 
framework was presented in [17]. The method forms a sample set of spatial information for each pixel. Then, 
the Spatial Sample Difference Consensus (SSDC) was computed for identification of foreground objects. An 
approach for detection of motion and segmentation was developed in [18]. The approach controls the 
uncertainties in the movement of the camera and computational disparities. By combining the color, motion 
probability and depth cues, the approach has segmented the moving objects. With importance to statistical 
learning, an efficient method for image segmentation was presented in [19]. The method applies the Rayleigh 
distribution to compute the probability density of background pixel. This method was applied to images of 
various colors. An algorithm based on the scale invariant feature transform (SIFT) was contributed in [20]. 
The method was applied for identifying gradual transitions and sudden variations without a need for training 
of the video. A method for detection of moving objects in GPU was proposed in [21]. This method has 
enhanced the quality of outcome in scenarios where the background and the objects in motion appear to be 
same. This method has automatically chosen the regions of interest (ROI). 


2. RESEARCH METHOD 

Two algorithms FDNF-HSV and FDNF-G are proposed for foreground movement identification. 
The FDNF-HSV algorithm identifies the foreground movement using HSV color model. Section 2.1.1 
presents the algorithm for FDNF-HSV. Section 2.2.1 presents the algorithm for FDNF-G. The algorithms 
common for FDNF-HSV and FDNF-G (i.e.) calculation of threshold and drawing bounding box are 
presented in sections 2.1.2 and 2.1.3 respectively. 


2.1. Motion detection using HSV channel 

The proposed method referred as FDNF-HSV (Frame Difference between Neighboring Frames 
using HSV) focuses on the foreground detection. The proposed algorithm works as follows: Using HSV 
(Hue, Saturation, Value) color model, a pixel-based difference among the frames are computed. For each 
frame taken into account, the three components of the frame (i.e.) HSV are obtained. Based on the HSV, the 
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difference among the successive frames (i.e.) deln, del; and del, are computed. Subsequently, three binary 
images are obtained by applying the proposed adaptive threshold algorithm presented in 2.1.2. Using 
equation (1), the resultant image is obtained by calculating the maximum value from each pixel of binary 
images. 


result, =max(th _im,th _im,,th _im,) (1) 


After filtering the small connected regions in the results, the proposed algorithm detects the 
foreground movement by applying the bounding box method. 


2.1.1.Algorithm for Motion Detection using HSV 
Step 1: In, = frameRGBtoHSV (b) //changes the true color image RGB to the HSV of background image b 
Step2 :for f=2toNF-1 //NF denotes the number of frames 


In ¢ = frameRGBIoHSV (f ) Ing = {In(i, j)|1<i<row,1< j <col} 
Step i (i.e.) 
// changes the true color image RGB to HSV color channels 
In = frameRGBtoHSV | f +1 
Step ii: Similarly fh ( ) 
Step iii: Find the absolute difference between H channel In p+ and In f 
del, =| mnga D-n; C, 1)| 
Step iv : Find the absolute difference between S channel In f+ and In f 
del, =| Mp 62- Ings D| 


Stepv : Find the absolute difference between V channel /n pei and In f 


del, =| My 653) Ing, 3)| 
Step vi : Call th_im = Threshold( del,, del, ) 
Step vi : Call th_iml = Threshold( del,, del, ) 
Step vii : Call th_im2 = Threshold( del,, del, ) 
Step viii: Find the result by finding pixel-wise the maximal value 
result r= max(th _im,th _im,,th_im,) 
Step ix : Convert image result to Boolean image by applying 
So T : result; (i, j) =1 
bool (i, j) = 
F : result , (i, j) =0 


Visisrow,1< j< col 


Stepx : Draw bounding box to detect the foreground movement. 
2.1.2.Algorithm for Threshold calculation: Threshold (im, , im, ) 


Step 1 : Calculate threshold using © = > max (im, ) + > max 


| (e) 
row + col \col col row row 


Step2 :If © #0 then binary image obtained is 
timage ; = {pix(i j) |l<i<row,l<j< col , where 
pir(i,j) = f Vim, (i, j) € im :im (i, j) < °| 
1, otherwise 
Step 3 : If © —O then binary image obtained is 
timage ¢ = { pix(i, j) |l<i<row,l<j< col} , where 
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pix(i, j) = fim, (i, j)|1<is< row,1< j <col} 


2.1.3.Algorithm for Drawing Bounding Box 

Step 1 : Find the connected regions in the given Boolean image. 
Step 2 : Filters out smaller connected regions 

Step 3 : Identify the four corners for each region, i.e. blobs 

Step 4 : Mark blobs found in the frame with rectangles. 


2.2. Motion Detection using Grey scale 

The proposed algorithm termed as FDNF-G (Frame Difference between Neighboring Frames using 
Grey scale) combines the background subtraction with the pixel-based variation between neighboring frames. 
The algorithm starts by finding the grey scale for the frames in the video sequence under consideration. Three 
different images del; (absolute difference between the current frame and its previous frame), delz (absolute 
difference between the current frame and its next frame) and del3(absolute difference between the current 
frame and the background frame) are obtained. By applying the proposed adaptive threshold algorithm 2.1.2, 
the binary images for del), delz and delz are obtained. After filtering the noise, the algorithm identifies the 
foreground motion. Algorithm for motion detection using greyscale: 


Step 1: In, = frameRGBtoGrey (b) // changes the true color image RGB to the grayscale intensity of 


background image b 
In, bin, 
Step 2: Find binary image for as 
Step2 : for f=2toNF-1  // NF denotes the number of frames 


Ins = frameRGBtoGrey (f ) me = {in(i, j) |l<i<row,l<sj< col} 
2 (i.e.) 


color image RGB to the grayscale intensity of frames. 


Step i // changes the true 


eee ver mfa = frameRGBtoGrey ( f = 1) M fy = frameRGBtoGrey( f + 1) 
tepii : Similarly 
Step iii : Find the absolute difference of In, and In, : del, =| Insa — In ¢ | 
iv : Find the absolute diff T 
Step iv : Find the absolute difference o an : del, =| In pay -Inş | 


Step v : Find the absolute difference of Jn, and In, : del, =| In, — In f | 


Step vi : Call th_im = Threshold( del, ,bin, ) 

Step vii : Call th_im1 = Threshold(del,,th _im) 

Step viii: Call th_im2 = Threshold( del,,th _im ) 

Step ix : Find the result by finding pixel-wise logical or operation 
result, =th _im |th_im |th_im, 


Step x : Convert image result , to Boolean image by applying 


; i(i ) T : result; (i, j)=1 
NT F : result, (i, j) =0 


Visisrow,1< j <col 
Step xi : Draw bounding box to detect actor 


3. RESULTS AND DISCUSSION 

The proposed algorithms FDNF-HSV and FDNF-G have been assessed with the datasets viz. 
UCSD, I2R and CAVIAR. The outcome of the assessments is shown in Figure 1. The proposed algorithm 
was assessed with various metrics viz. recall, precision, F-measure and percentage of wrong classification 
(pwc). The following parameters were considered for computing these metrics. 
a. True positive (TP)-pixels in a foreground object categorized as pixels in the foreground. 
b. True negative (TN)-pixels in a background object categorized as pixels in the background. 
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c. False positive (FP)-pixels in a background categorized as pixels in the foreground. 
d. False negative (FN)-pixels in a foreground categorized as pixels in the background. 


Figure 1. Sample output for the UCSD dataset, I2R dataset and CAVIAR dataset 


The calculation of pwc, precision, recall and F-measure (similarity measure) are given here. 


FN + FP TP TP 
pwe = x 100% pre = ———— recall = ————— 
TP + FN + FP +TN TP + FP TP + FN 


2x recall x pre 


Fmeasure = 


recall + pre 


With reference to I2R, CAVIAR and UCSD datasets, the proposed algorithms FDNF-HSV and 
FDNF-G detect activities with complex differences in the background. This encompasses illumination 
variations and acquiring dynamic activities. The I2R dataset comprises video sequences for identification of 
foreground objects with intricate background. Pertinent to F-measure metric for the I2R dataset, an 
assessment of the proposed approach (FDNF-HSV) with other techniques is presented in Table 1. Results 
exhibit that, the proposed approach has accomplished maximum F-measure than the other techniques. 
Figure 2 provides a graphical representation of the results attained for depicting the metrics viz., precision, 
recall, F-measure for I2R dataset. Figure 3 presents a graphical representation of the outcome obtained for the 
metrics viz., precision, recall, F-measure for CAVIAR dataset. 


Table 1. Assessment of F-measure with various techniques and the proposed method (FDNF-HSV) 


for the I2R dataset 
Bootstrap Escalator Fountains Lobby Water surface 
FrmDiff[1] 35.36 21.72 25.43 16.33 24.26 
MoG[6] 56.29 41.17 76.91 47.92 79.7 
ACMMM03[5] 60.43 32.6 56.51 30.31 63.66 
Code Book [7] 63.66 49.82 61.36 25.51 73.09 
SILTP[8] 73.32 65.88 86.23 78.57 84.36 
ViBe[9] 78.26 64.72 60.96 26.55 86.82 
CS-SILTP[3] 76.35 70.72 87.46 80.23 87.38 
BITC[10] 64.86 63.37 95.24 76.67 93.02 
DFB —A[4] 71.86 66.37 77.43 13.24 93.81 
Non-Parametric[21] 64.10 - 70.49 - 90.11 
Proposed Approach 
( FDNF-HSV) 86.48 78.04 86.69 97.43 98.98 
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I2R DATASET 


Bootstrap Esculator Fountains Lobby WaterSurface 


=¢= PRECISION = RECALL © e#e FMEASURE 


Figure 2. Graph depicting metrics viz., precision, recall, F-measure for I2R dataset 
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Figure 3. Graph representing metrics viz., precision, recall, F-measure for CAVIAR dataset 


The proposed method (FDNF-G) aims at identifying foreground movement in video sequences 
available in UCSD background subtraction dataset. The results of validation are presented in Table 2. Results 
illustrate that, the proposed method has attained an improved F-measure when compared with other methods 
taken into account. 


Table 2. Comparison of the proposed technique (FDNF-G) with various methods on UCSD background 
subtraction dataset with regard to average F-measure 


Sequences FrmDiff MoG ACMMM03 Code Book SILTP ViBe CS- Proposed Work 
[1] [6] [5] [7] [8] [9] SILTP [B] (FDNF-G) 

Birds 19.63 28.35 28.82 2121 30.11 31.92 29.76 44.44 
Bottle 19.31 51.12 23.44 23.09 55.23 57.5 63.41 100 
Freeway 26.81 53.06 31.88 38.48 51.94 54.62 55.54 86.41 
Ocean 10.07 28.95 19.03 22 57.87 26.33 60.63 94.11 
Pedestrians 24.28 79.63 28.89 64.3 81.09 80.1 85.7 94.73 
Rain 41.88 74.81 81.28 43.28 85.24 93.7 89.74 94.34 
AvgF- 23.66 52.65 35.56 35.39 60.25 57.36 64.13 85.67 


measure 


4. CONCLUSION 

Research on various methods and algorithms in video processing is commendable. Relevant to the 
identification of foreground in video sequences, lot of algorithms are available in the literature. Most of these 
algorithms have done the foreground identification with less attention to variations in illumination and 
background. But, the proposed algorithms FDNF-HSV and FDNF-G have identified the foreground 
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movement with importance to changes in illumination and background. The proposed work has performed 
foreground movement identification by calculating the adaptive threshold of the variations between frames. 
Results of the proposed algorithms are better than the other standard algorithms in terms of similarity 
metrics. 
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